1. Field of the Invention
This invention relates to an image acquiring method and apparatus, wherein an image is picked up through checkered sampling or square sampling, and an image signal representing the image is thereby acquired. This invention also relates to an image processing method and apparatus, wherein sharpness enhancement processing is performed on the image signal. This invention further relates to an image transforming method and apparatus, wherein transforming processing is performed on a square sampling image signal, which has been obtained by transforming a checkered sampling image signal. This invention still further relates to an image transforming method and apparatus, wherein a checkered sampling image signal is transformed into a square sampling image signal. This invention also relates to a recording medium, on which a program for causing a computer to execute one of the methods has been recorded and from which the computer is capable of reading the program.
2. Description of the Related Art
With digital electronic still cameras (hereinbelow referred to as the digital cameras), an image having been acquired from a photographing operation is stored as a digital image signal on a recording medium, such as an internal memory located within the digital camera or an IC card. Image processing, such as gradation processing or sharpness enhancement processing, may then be performed on the digital image signal to obtain a processed image signal, and the thus obtained processed image signal may be utilized for reproducing the image, which was acquired from the photographing operation, as a hard copy, such as a print, or a soft copy on a display device. In cases where the images having been acquired with the digital cameras are reproduced in the manner described above, it is desired that images having image quality as good as the image quality of photographs printed from negative film be obtained. Therefore, a novel image reproducing technique has been proposed in, for example, Japanese Unexamined Patent Publication No. 10 (1998)-191246. With the proposed image reproducing technique, information giving specifics about a photographing operation, such as the presence or absence of strobe light or the kind of illumination utilized at the time of the image acquisition, is appended to an image signal having been acquired with a digital camera, and the image signal appended with the information giving specifics about the photographing operation is fed out from the digital camera. Also, with the proposed image reproducing technique, when image processing is to be performed on the image signal, reference is made to the information giving specifics about the photographing operation, which information has been appended to the image signal, and appropriate image processing is performed on the image signal.
In order for an image signal to be acquired with digital cameras or scanners, it is necessary for the image to be picked up through sampling in a predetermined sampling pattern. As the sampling in a predetermined sampling pattern for the image acquisition, checkered sampling has heretofore been known. FIG. 3 is an explanatory view showing a pixel array of pixels represented by image signal components of a checkered sampling image signal, i.e. an image signal acquired through checkered sampling. With the checkered sampling, the image is acquired such that the pixel positions in the pixel array shown in FIG. 3, which are indicated by the “∘” mark, have signal values, and the pixel positions indicated by the “x” mark (hereinbelow referred to also as the empty pixel positions) have no signal value. FIG. 18A is an explanatory view showing a pixel array employed in checkered sampling. Actually, in the checkered sampling, pixels are arrayed in the pattern shown in FIG. 18A. FIG. 18B is an explanatory view showing a pixel array employed in square sampling. When the pixel array shown in FIG. 18A and the pixel array shown in FIG. 18B are compared with each other, the pixel density in the pixel array shown in FIG. 18A becomes higher than the pixel density in the pixel array shown in FIG. 18B. Therefore, in cases where the area for the image pickup is the same, with the pixel array shown in FIG. 18A, a larger amount of information can be obtained than with the pixel array shown in FIG. 18B. A checkered sampling image signal, i.e. an image signal acquired through the checkered sampling, can be obtained by, for example, performing the photographing operation with a digital camera provided with a charge coupled device (CCD) image sensor, which comprises photoelectric conversion devices arrayed in a checkered pattern. The checkered sampling image signal can also be acquired with a digital camera provided with a CCD image sensor, which comprises photoelectric conversion devices arrayed in a square pattern, by obtaining a first image signal with a single photographing operation, then rotating the CCD image sensor to an oblique direction by an angle of 45 degrees, and obtaining a second image signal in this state. The checkered sampling image signal can further be acquired by sampling the image signal components of an image signal, which has been obtained with a CCD image sensor comprising photoelectric conversion devices arrayed in a square pattern, in a checkered sampling pattern.
However, image output devices for outputting images on image output media, such as a cathode ray tube (CRT) monitor or printing paper, are designed so as to process a square sampling image signal, i.e. an image signal acquired through the square sampling, which has signal values at all of the pixel positions indicated by the “∘” mark and the “x” mark in the pixel array shown in FIG. 3. Also, image processing units for performing the image processing, such as the sharpness enhancement processing, on image signals are designed so as to process square sampling image signals. Particularly, in the sharpness enhancement processing, processing appropriate for the square sampling image signal is performed, wherein the processing with respect to the vertical direction of an image and the processing with respect to the horizontal direction of an image can be separated from each other. Therefore, heretofore, with respect to the checkered sampling image signal, signal values corresponding to the pixel positions indicated by the “x” mark are calculated by performing interpolating operations on the signal values corresponding to the pixel positions indicated by the “∘” mark, the checkered sampling image signal is thereby transformed into the square sampling image signal, and thereafter the image output or the image processing is performed in accordance with the thus obtained square sampling image signal.
In cases where frequency characteristics of the checkered sampling image signal are represented in two-dimensional directions, the frequency characteristics may be represented as a rhombic response illustrated in FIG. 19A. Also, the frequency characteristics of the square sampling image signal may be represented as a rectangular response illustrated in FIG. 19B. In FIG. 19B, fs/2 represents a Nyquist frequency of the square sampling image signal. Also, in FIGS. 19A and 19B, image-reproducible frequency bands are indicated by the solid lines. In cases where the checkered sampling image signal is transformed into the square sampling image signal, the frequency characteristics, which express the original image information, do not alter from the frequency characteristics shown in FIG. 19A. Therefore, if the sharpness enhancement processing, which can be separated with respect to the vertical direction and the horizontal direction of the image, is performed on the checkered sampling image signal in the same manner as that for the square sampling image signal, the frequency bands originally having no image information, which are indicated by the hatching in FIG. 19B, will be enhanced, and therefore noise will occur. As a result, the image quality of an image obtained by performing the processing will become bad.
As a technique for transforming a checkered sampling image signal to a square sampling image signal, a technique has been proposed in, for example, Japanese Unexamined Patent Publication No. 5 (1993)-145857, wherein microlenses arrayed in a checkered pattern are located at a stage prior to a CCD image sensor in a camera, a filtering process is performed on signal values, which have been obtained in the checkered pattern from the CCD image sensor, by use of a linear interpolation filter illustrated in FIG. 12 such that a mean value of the signal values arrayed in the vertical and horizontal directions may be calculated, and a signal value corresponding to each of the pixel positions indicated by the “x” mark in FIG. 3 is thereby calculated.
However, with the technique proposed in Japanese Unexamined Patent Publication No. 5 (1993)-145857, the simple mean value of the signal values arrayed in the vertical and horizontal directions at each empty pixel position indicated by the “x” mark in FIG. 3 is calculated with the interpolation filter having approximately rhombic characteristics shown in FIG. 12, and the signal value corresponding to the empty pixel position is thereby calculated. Therefore, though the information of the checkered sampling image signal within the Nyquist frequency can be kept, the problems occur in that the image represented by the thus obtained image signal becomes unsharp. Accordingly, even if image processing is performed on the thus obtained image signal or a visible image is reproduced from the thus obtained image signal, an image having good image quality cannot always be obtained.